COMPACT INTEGRATED SELF CONTAINED SURVEILLANCE UNIT 
Field of the Invention 

[00001] The invention relates to an integrated, self contained surveillance system. 
More specifically, this invention relates to a programmable, multi-mode surveillance 
5 system which incorporates modular information fusion from multiple sources to perform 
multiple functions. The system is compact and light weight, making it ideal for man- 
portable remote operations. 
Background of the Invention 

[00002] U.S. patent No. 5,864,481 to M.D. Gross et al. describes an integrated 

10 reconfigurable man-portable communication system, adapted particularly for use in 
military or law enforcement tactical situations. The system includes a computer with 
mission oriented hardware and software, which provides the focal point for integration of 
various peripheral components. The reconfigurable man-portable system also includes a 
ballistic helmet with an integrated audio headset assembly, a weapon mounted thermal 

15 weapons site, a remote video system, and a laser rangefmder/digital compass assembly. 
The helmet itself has a modular heads-up display assembly for providing remote viewing 
via a helmet mounted laser detection system. The heads-up display assembly and the 
audio headset assembly provide the user with tactical information such as 360 degree 
laser detection from helmet mounted detectors, target location for direct and indirect 

20 target engagement from the weapon mounted laser rangefmder/digital compass assembly, 


navigational positioning from a global positioning system, and an individual radio 
module integrated within a modular load carrying device. 

[00003] As is evident from the specification and drawings, the system of M.D. 
Gross et. al. is bulky and cumbersome. The system consists of a number of major 
5 subsystems including: (1) a combined computer and radio subsystem which, along with 
batteries and other equipment, is carried on a specially designed pack frame; (2) weapons 
systems; (3) a helmet assembly subsystem; and (4) what is termed a "remote input 
pointing device." The separate weapons subsystems, which are each carried on the 
weapon (e.g., Ml 6), include a thermal weapons sight, video camera, laser 

10 rangefmder/digital compass, and an infrared aiming light. All of these components are 
separately housed and separately mounted on the weapon. The helmet assembly includes 
a specially designed (or specially modified) helmet incorporating the heads-up display 
and the plurality of laser detectors. The foregoing components are interconnected by 
wiring, some of which is exposed. 

15 [00004] U.S. patent No. 6,020,994 to L.G. Cook discloses a system and 

mechanical packaging for what is described as a fiiUy integrated multifrmctional 
multispectral sight assembly. The packaging permits the use of the sight on hand-held, 
man-portable units. All the subsystems of the sight assembly (a direct view day 
subsystem, a CCD-based visible TV subsystem, a staring IR night subsystem, an eye safe 

20 laser rangefinding subsystem and a display subsystem for viewing the output of the TV or 
IR subsystem) are aligned and use a single telescope and eye piece in order to decrease 
the size of the assembly. In addition, each sight assembly subsystem operates in a first 
and second field of view, preferably in a narrow field of view and a wide field of view. 
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[00005] U.S. patent No. 6,144,03 1 to R. J. Herring et. AL discloses a video camera 
system with an uncooled focal plane array and radiation shield. 
Objects of the Invention 

[00006] It is an object of the present invention to provide a programmable, multi- 
5 mode surveillance system. 

[00007] It is an additional object of the present invention to provide a multi-mode 
surveillance system which incorporates information fusion from multiple sources to 
perform multiple functions. 

[00008] It is an additional object of the present invention to provide such a multi- 
10 mode surveillance unit in a tightly integrated package to provide a unit having both low 
weight and volume. 

[00009] It is a further object of the present invention to provide a multi-mode 
surveillance system which can be utilized either remotely or hand carried by its user. 
[00010] It is yet a further object of the present invention to provide a surveillance 
15 system which, because of its compact design and low weight, is especially suitable for 
man-portable remote locations. 

[00011] When hand carried by its user, it is yet another object of the present 
invention to provide the user with a compact, cabled and wearable heads-up display 
(HUD) to connect the user with the surveillance system's information display. 
20 [00012] It is yet a further object of the present invention to provide a surveillance 

unit which has targeting capabilities and which can be moxmted on a weapon such as an 
M4,M16, or AR15. 

3 


[00013] It is yet a further object of the present invention to provide a surveillance 
unit which includes target auto-tracking. 

[00014] It is yet a further object of the present invention to provide a surveillance 
unit which includes ballistics computation. 
5 [00015] It is still yet a further object of the present invention to provide a compact 

surveillance unit which incorporates, inter alia, satellite (and airborne) imagery, GPS 
and/or GLONASS position measurement, rangefmding, ballistics computation, secondary 
voice communications, and temperature sensing, 

[00016] It is still another object of the present invention to provide a compact 
10 surveillance system which incorporates sensor tracking (e.g., GPS for slow position 
measurement, an accelerometer for fast time position measurement). 

[00017] It is yet another object of the present invention to provide a surveillance 
system which incorporates power management. 

[00018] It is still yet another object of the present invention to provide a compact 
15 surveillance system which incorporates event logging (e.g., message sent, message 
received, rounds fired, unit turned off, unit turned on, change in position, change in 
position of an object under surveillance). 

[00019] It is still yet another object of the present invention to provide a 
surveillance system which uses off-the-shelf components (e.g., compass, card slots, touch 
20 pads, optics, video jjP, programmable memory, etc.). 

[00020] Other objects, advantages and novel features of the present invention will 
become apparent jfrom the following detailed description, considered in conjunction with 
the accompanying drawings. 
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Summary of the Invention 

[00021] An integrated, compact, self contained surveillance unit including: a 
housing having a front end and a back end; a sensor section attached to the front end of 
5 the housing; mechanisms supported by the housing for determining the position of the 
unit; a communications system supported by the housing; a computer supported by the 
housing; and image output attached to the back end of the housing connected to the 
computer. The computer is connected to the sensor section, the position determination 
mechanisms, the communication system and the image output. The sensor is selected 

10 from the group including visible sensors, UV sensors, short v^avelength infrared sensors 
and long wavelength infrared sensors. The long wavelength sensor section also includes 
an uncooled focal plane array. The position determination mechanisms include GPS, an 
accelerometer and a compass. The communication system, which is bi-directional, 
includes a multi-mode patch antenna and a software definable radio (SDR). The 

15 computer includes digital signal processing, field programmable gate array and memory 
(both volatile and non- volatile). The surveillance unit also includes a temperature sensor 
connected to the computer. The image output is, preferably, a heads up display which is 
detachable from the housing. Finally, the surveillance unit includes mechanisms for 
inputting and manipulating data. The information manipulating mechanism includes a 

20 touchpad. 

Brief Description of the Drawings 

[00022] Figure 1 is a front perspective view of the compact, integrated, self- 
contained surveillance system of the present invention; 

[00023] Figure 2 is a back perspective view of the surveillance system of Figure 1 ; 
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[00024] Figure 3 is a back view; and 

[00025] Figure 4 is a schematic, which illustrates the data flow between the 
subsystems of the surveillance system and with external systems. 
Description of the Preferred Embodiment 
5 [00026] Surveillance system 1 1 is divided basically into three sections: (1) optics 

section 13; (2) electronics and power section 15; and (3) heads up display (sometimes 
referred to as eyepiece) 19. Electronics and power section 15 are supported by housing 
21, which includes a mechanical attachment section (not shown) to secure unit 1 1 to the 
mounting rail incorporated on small arms, such as the Ml 6, M4 and AR15. 

10 [00027] Housed within optics section 13 are the system's optics 23, such as a long 

wave infrared sensor (LWIR), uncooled focal plan array 25 and a video |jP 27. See 
Figure 4. Within section 15 are supported battery 17, antenna 29, radio frequency 
transceiver 31, GPS/GLONASS position measurement unit 33, compass 35, digital signal 
processor (DSP)/field programmable gate array (FPGA) 37, memory 39 and 

15 accelerometer 41. Optionally, section 15 can also include temperature sensor 43. 

Housing 15 also includes a Type II PC card slot 45, an external USB data port 47, and a 
touch pad 49. 

[00028] With reference to Figure 4, optics 23 is coupled to focal plane array 25 
(such as an uncooled infrared microbolometer focal plane array manufactured by The 
20 Boeing Company). Video |JP 27 is coupled to the intemal electronics of focal plane array 
25 to permit the output of data from optics 23. 

[00029] Connection 59 couples video |jP 27 with DPS/FPGA 37. Via connection 
59 and video |jP 27, DSP/FPGA 37 sends a control signal to focal plane array 25 to read 
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out specific pixels (included in the focal plane array) for purposes of calibrating focal 
plane array 25 and the gains and offsets used to correct for focal plane anomalies, (Since 
the calibration process also includes sensing ambient temperature, focal plane array 25 
can also be used as a temperature sensor, thus eliminating the need for temperature sensor 
5 43.) In response to a command from DSP/FPGA 37, internal electronics reads a specific 
set of pixels and sends the corrected data, including temperature, back to DSP/FPGA 37 
via video |JP 27 and connection 59. 

[00030] The corrected streaming video from video |jP 27 is then sent to 
DSP/FPGA 37, also over connection 59, which in turn sends the corrected streaming 

10 video (with or without enhancements, as explained below) to eyepiece 19 over 

connection 67. Eyepiece 19 is, preferably, a micro optical VGA heads up display which, 
in operation, can be detached from housing 21 via any conventional, mechanical 
connector (not shown). Though physically detachable, eyepiece 19 is still, via 
connection 67, connected to housing 21 and DSP/FPGA 37. 

15 [00031] Antenna 29 is, preferably, a conformal patch antenna which sends and 

receives RF signals. It has either a center frequency and bandwidth or, alternatively, a 
frequency range (preferably 1.2 to 2.4 GHz). Received RF signals are transmitted to 
GPS/GLONASS 33 via connection 71. RF signals are also transmitted to radio frequency 
transceiver 31, via connection 73. 

20 [00032] Radio frequency transceiver 3 1 is a software definable radio (SDR) which 

can be used as a cell phone, a satellite phone (in either case accelerometer 41, because of 
its sensitivity, is used as the speaker), a generic receiver (within the frequency range of 
the antenna 29), and as a generic transmitter (again within the frequency range of antenna 


29). Radio frequency transceiver 3 1 sends the base band information content of the RF 
signal to DSP/FPGA 37 over connection 75. Signals from DSP/FPGA 37 to radio 
frequency transceiver 31, to control its configuration and transmit data to a remote 
location, are also sent over connection 75. 
5 [00033] GPS/GLONASS 33 contains two chips v^hich receive the RF signals from 

antenna 29 and compute location based on the measurement of the time difference 
between RF signals from many different satellites. They provide parallel, redundant time 
keeping and user location fixnctions. The NMEA-0183 protocol is used for both. 
GPS/GLONASS 33 is connected to processor 37 via connection 77 which permits the 

10 transmission of signals to GPS/GLONASS 33 (where am I? time, or both) and data back 
(location, time, both). 

[00034] Compass 35 is a digital compass which sends a signal to DSP/FPGA 37 
via connection 8 1 . Temperature sensor 43 is optional, depending on whether focal plane 
array 25 is used to measure temperature. When incorporated, it is connected to 

15 DSP/FPGA 37 via connection 85 

[00035] Accelerometer 41 includes three independent micro electro mechanical 
structures (i.e., MEMS devices) which measure change of motion on three different axes 
(i.e., X, y and z). Each of the three MEMS devices independently sends, via connection 
83, a signal corresponding to a dc voltage to DSP/FPGA 37. Each signal ranges from 0 

20 (which represents the at rest position) to 2 volts (which represents full scale movement). 
Accelerometer 41 which, in the preferred embodiment is (approximately) 5mm x 5mm x 
1mm, is mounted on the interior side of housing 21. 
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[00036] The processor portion of DSP/FPGA 37 is, preferably, a 16 bit digital 
signal processor. Such a processor has a nxomber of features which make it desirable in 
the present invention (e.g., 9 different power management zones and multiple internal 
busses, which permits data to come in and out independently). Thus, the processor 
5 portion is capable of moving large amounts of data and, in parallel, simultaneously 
processing arrays of information and commands. This permits the real time fusion (i.e., 
combining) of different data from various inputs (e.g., preloaded information, video |jP 
27, antenna 29, accelerometer 41, etc.) and commands (as discussed below). The use of a 
DSP also has the advantage of conserving power, hence battery life is optimized. 

1 0 [00037] The FPGA within the DSP/FPGA combination provides for the 

programmable and reconfigurable aspects of the functionality of system 11. The FPGA 
off-loads from the DSP portion certain signal processing and data manipulation functions. 
It performs all functions significantly faster than can the DSP portion alone. It functions 
as a digital pre-, and post-, processor for a variety of data types (e.g. navigation, 

15 communication), the processing of large volumes of matrixed data (i.e. multiple images), 
and their filtering. User specific I/O functions (e.g., USB or 1394) can be programmed 
into the FPGA portion to allow users to integrate to their current system architectures 
without changing the hardware design. 

[00038] As previously indicated, housing 21 also supports a battery 17, card slot 

20 45, a touch pad 49, and a USB data port 47. Typically, card slot 45 is a Type II PC card 
slot which has a generic open spec. Alternately, where RF communications are not 
required, a 3COM type card slot can be used. For military applications a VDC 400 card 
slot can be used. Regardless of the type, a variety of pre-programmed data (depending 
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upon the specific application(s) for which system 1 1 is being used) is input to DSP/FPGA 
37 via card slot 45 and connection 91 . Touch pad 49, includes course (i.e., finger) and 
fine (e.g. stylus) manipulation controls. The finger control is used to control the cursor 
visible in the screen of heads up display 19. The stylus control can be used for inputting 
data (such as is done with a Palm Pilot) and handwriting recognition. Touch pad 49 is 
connected to processor 37 via link 93. USB data port 49, which is connected to 
DSP/FPGA 37 via link 92, is used to provide external command and control to/from 
other systems (e.g., a laptop computer). 

[00039] System 1 1 has three levels of software: applications program(s), an 
operating system; and a monitor code. The operating system is, for instance, EG 
Windows CE, VxWorks, which are real time operating systems which facilitate the use of 
the applications program(s). The monitor code provides core system connectivity with 
the DSP portion and aligns the operating system with the other components of system 1 1 
(e.g., GPS/GLONASS 33, radio frequency transceiver 3 1 , accelerometer 41, etc.). The 
applications code(s) runs on top of the operating system to perform the functions set forth 
herein. Connected to processor 37 via connection 95 is memory 39, which includes non- 
volatile type (e.g. flash RAM), which is not affected by power, and volatile, which is 
affected by power. The non- volatile memory contains all monitor code and core 
applications code. Volatile memory is provided for critical applications parameters. 

[00040] Finally, supported within housing 2 1 is power supply 1 7 which is 
connected to everything except antenna 29 and temperature sensor 43 (if provided). 
Typically, power supply 17 is a Li Ion battery. 
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[00041] Via heads up display 1 9, which is essentially a miniaturized computer 
screen, an array of data is selectively provided to the user. Inputs, via cable 67, include 
functional menus, a multi-mode cursor, navigational and positional images, streaming 
LWIR images, communications interface, passive rangefmding function, and ballistic 
solution crosshairs. With the foregoing both preprogrammed data and updated data can 
be viewed. The preprogrammed data can include terrain information, elevation data and 
3-D topographical data. It can also include mission tasking information. The updated 
data, which comes via antenna 29 (from, for instance, satellite or aircraft or another 
system 11), can include corrections, updates and additions to any and all of the foregoing. 
These sources of data can be viewed independently, or overlayed, or via split screen, in 
the screen of heads up display 1 9 at the discretion of the user via touch pad 47. For 
instance, the LWIR corrected video stream can be overlayed with GPS information, 
temperature and crosshairs. As previously mentioned, use of touch pad 47, via one's 
finger, controls the cursor visible on the screen to select the desired function from the 
various menus. The floating crosshairs are, in conjunction with accelerometer 41, used 
for ballistics computations (except for windspeed), automatic target tracking and passive 
range finding. 

[00042] In operation, system 1 1 can be hand carried by an operator and used 
during daylight and at night. It enhances night vision while providing a host of automatic 
user location functions. The GPS and/or GLONASS 33 provide the user with global 
location to within 6m, while accelerometer 41 and compass 35 combination measures the 
users change in location and his/her direction. DSP/FPGA 37 can combine these 
functions and add functionality by comparing two images of the same target taken at 
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different angles , compare the motion of system 1 1 between images to derive range to a 
point in the image, and determine its exact location as well. In fixed applications, 
processor 37 can infer range to a moving object by pattern recognition (e.g. a vehicle 
passes in front of a light pole or the lines in a parking lot). Once a distance is estimated a 
5 velocity and direction can be inferred. A message can be sent to another monitoring 
location or, say, law enforcement officials, concerning these measurements. In fixed 
installation GPS/GLONASS 33 would be unnecessary. 

[00043] Many modifications and variations of the present invention are possible in 
light of the above teachings. It is therefore to be understood that, within the scope of the 
10 appended claims, the invention may be practiced otherwise than as specifically described. 


12 


